A presentation for Research Data at the Edge, Day One of Duke Research Computing Symposium
Why?
- The Web has lots of stuff
- frontier beyond curated datasets
- stuff is wrapped in HTML
- The HTML is transported over HTTP but composed for h2m consumption
- Intellectual Property Rights bear serious consideration
API
Application Program Interface
- Built for machine-to-machine interactions
- Instructions for programs
Client / Server
- Make [R] interface with the web, same as h2m but now m2m
Human Simulation
- Human version
- Person uses Web Client
- Person enters a URL
- client & server negotiate

- Information is sent back in wrapped HTML
- Web Browser parses the HTML
JSON
- Javascript Object Notation is a language-independent data format
- Currently the most common data data format for asynchronous client/server communication format
- Consists of key-value pairs
# from https://en.wikipedia.org/wiki/JSON
{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
},
{
"type": "mobile",
"number": "123 456-7890"
}
],
"children": [],
"spouse": null
}
LS0tDQp0aXRsZTogIlVzaW5nIFIgdG8gT3JjaGVzdHJhdGUgQVBJcyINCmF1dGhvcjogIkpvaG4gTGl0dGxlIg0KZGF0ZTogJ2ByIFN5cy5EYXRlKClgJw0Kb3V0cHV0Og0KICBzbGlkeV9wcmVzZW50YXRpb246IGRlZmF1bHQNCiAgaHRtbF9ub3RlYm9vazogZGVmYXVsdA0KLS0tDQpBIHByZXNlbnRhdGlvbiBmb3IgW1Jlc2VhcmNoIERhdGEgYXQgdGhlIEVkZ2VdKGh0dHA6Ly9saWJyYXJ5LmR1a2UuZWR1L2VkZ2UvZXZlbnRzL3JjMTcpLCBEYXkgT25lIG9mIFtEdWtlIFJlc2VhcmNoIENvbXB1dGluZyBTeW1wb3NpdW1dKGh0dHBzOi8vcmMuZHVrZS5lZHUvc3ltcG9zaXVtLTIwMTcvKQ0KDQojIyBPdXRsaW5lDQoNCiogQVBJDQoqIEpTT04NCiogUiAvIFJTdHVkaW8NCg0KIyMgV2h5Pw0KPCEtLSBOQVNBIGFuaW1hdGVkIEdJRiAvLy8gIGh0dHA6Ly9pLmdpcGh5LmNvbS9sMkpodDRsSWZFUWZKM3pqMi5naWYgICAgLS0+IA0KPCEtLSAgZ29vZCBodW1hbiBoYW5kc2hha2UgLy8vICBodHRwOi8vZ2lwaHkuY29tL2dpZnMvdGhvbWFzLVUyWGJvUnVOODlJZGkgLS0+DQo8IS0tIGFmdGVyIHRoZSByZXNlYXJjaCBoYW5kc2hha2UgaXMgY29tcGxldGUgLy8vIGh0dHA6Ly9naXBoeS5jb20vZ2lmcy84MHMtMTk4MHMtdGhvbWFzLWRvbGJ5LXdDS21CZDdvTnRBNGcgIC0tPiANCjwhLS0gdGhlIGNvbmZ1c2lvbiBvZiB0aGUgbTJtIGhhbmRzaGFrZSAvLy8gICBodHRwOi8vZ2lwaHkuY29tL2dpZnMvdGhvbWFzLU1qa0NZak00Nk5yck8gLS0+DQoqIFRoZSBXZWIgaGFzIGxvdHMgb2Ygc3R1ZmYNCiAgICArIGZyb250aWVyIGJleW9uZCBjdXJhdGVkIGRhdGFzZXRzDQogICAgKyBzdHVmZiBpcyB3cmFwcGVkIGluIEhUTUwNCiAgICArIFRoZSBIVE1MIGlzIHRyYW5zcG9ydGVkIG92ZXIgSFRUUCBidXQgY29tcG9zZWQgZm9yIGgybSBjb25zdW1wdGlvbg0KICAgICsgSW50ZWxsZWN0dWFsIFByb3BlcnR5IFJpZ2h0cyBiZWFyIHNlcmlvdXMgY29uc2lkZXJhdGlvbg0KDQojIyBBUEkNCkFwcGxpY2F0aW9uIFByb2dyYW0gSW50ZXJmYWNlIA0KDQoqIEJ1aWx0IGZvciBtYWNoaW5lLXRvLW1hY2hpbmUgaW50ZXJhY3Rpb25zDQoqIEluc3RydWN0aW9ucyBmb3IgcHJvZ3JhbXMNCg0KPCEtLSBodHRwOi8vbW9iaWxlLWdwcy5uZXQvMjAxNS8wMS8gLS0+DQohW10oaHR0cDovL21vYmlsZS1ncHMubmV0L2dwcy1zeXN0ZW1zLXdpdGhvdXQtYXBpLykNCg0KDQotLS0gICAgDQoNCiMjIyBDbGllbnQgLyBTZXJ2ZXIgDQo8IS0tIGh0dHBzOi8vcGl4YWJheS5jb20vZW4vY2xpZW50LXNlcnZlci1uZXR3b3JraW5nLWxhcHRvcC0zNDE0MjAvIC0tPg0KDQohW10oaHR0cHM6Ly91cGxvYWQud2lraW1lZGlhLm9yZy93aWtpcGVkaWEvY29tbW9ucy90aHVtYi9jL2M5L0NsaWVudC1zZXJ2ZXItbW9kZWwuc3ZnLzUwMHB4LUNsaWVudC1zZXJ2ZXItbW9kZWwuc3ZnLnBuZykgDQoNCiogTWFrZSBbUl0gaW50ZXJmYWNlIHdpdGggdGhlIHdlYiwgc2FtZSBhcyBoMm0gYnV0IG5vdyBtMm0NCg0KLS0tICANCg0KIyMjIEh1bWFuIFNpbXVsYXRpb24NCiogSHVtYW4gdmVyc2lvbg0KICAgICsgUGVyc29uIHVzZXMgV2ViIENsaWVudA0KICAgICsgUGVyc29uIGVudGVycyBhIFVSTDxicj4NCiAgICAhW10oaHR0cHM6Ly91cGxvYWQud2lraW1lZGlhLm9yZy93aWtpcGVkaWEvY29tbW9ucy9kL2Q2L1VuaWZvcm1fUmVzb3VyY2VfTG9jYXRvcl8lMjhVUkwlMjlfZXhhbXBsZS5QTkcpDQogICAgPCEtLSBodHRwczovL2NvbW1vbnMud2lraW1lZGlhLm9yZy93aWtpL0ZpbGU6VW5pZm9ybV9SZXNvdXJjZV9Mb2NhdG9yXyhVUkwpX2V4YW1wbGUuUE5HIC0tPg0KICAgICsgY2xpZW50ICYgc2VydmVyIG5lZ290aWF0ZTxicj4gDQogICAgIVtkcmFtYXRpemF0aW9uOiBnb29kIGhhbmRzaGFrZV0oaHR0cDovL2kuZ2lwaHkuY29tL1UyWGJvUnVOODlJZGkuZ2lmKSANCiAgICArIEluZm9ybWF0aW9uIGlzIHNlbnQgYmFjayBpbiB3cmFwcGVkIEhUTUwNCiAgICArIFdlYiBCcm93c2VyIHBhcnNlcyB0aGUgSFRNTCANCiAgICA8IS0tIGh0dHBzOi8vY29tbW9ucy53aWtpbWVkaWEub3JnL3dpa2kvRmlsZTpIVE1MLnN2ZyAtLT4NCg0KDQoNCg0KIyMgbTJtIC0tIGRldmVsb3BtZW50DQoNCg0KIVtkcmFtYXRpemF0aW9uOiBjb25mdXNlZCBhYm91dCB0aGUgcHJvdG9jb2xdKGh0dHA6Ly9pLmdpcGh5LmNvbS9NamtDWWpNNDZOcnJPLmdpZikNCiAgICANCiMjIEpTT04NCg0KKiBbSmF2YXNjcmlwdCBPYmplY3QgTm90YXRpb25dKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL0pTT04pIGlzIGEgbGFuZ3VhZ2UtaW5kZXBlbmRlbnQgZGF0YSBmb3JtYXQNCiogQ3VycmVudGx5IHRoZSBtb3N0IGNvbW1vbiBkYXRhIGRhdGEgZm9ybWF0IGZvciBhc3luY2hyb25vdXMgY2xpZW50L3NlcnZlciBjb21tdW5pY2F0aW9uIGZvcm1hdA0KKiBDb25zaXN0cyBvZiBrZXktdmFsdWUgcGFpcnMNCg0KPCEtLSBodHRwOi8vaS52aW1lb2Nkbi5jb20vdmlkZW8vNTQxOTM1ODE2XzEyODB4NzIwLmpwZyAtLT4NCjwhLS0gVmltZW8gb24gV2hhdCBpcyBKU09OIC8vIGh0dHBzOi8vdmltZW8uY29tLzE0NDE2MjEwMiAtLT4NCg0KDQpgYGB7anNvbiBleGFtcGxlfQ0KIyBmcm9tIGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL0pTT04NCnsNCiAgImZpcnN0TmFtZSI6ICJKb2huIiwNCiAgImxhc3ROYW1lIjogIlNtaXRoIiwNCiAgImlzQWxpdmUiOiB0cnVlLA0KICAiYWdlIjogMjUsDQogICJhZGRyZXNzIjogew0KICAgICJzdHJlZXRBZGRyZXNzIjogIjIxIDJuZCBTdHJlZXQiLA0KICAgICJjaXR5IjogIk5ldyBZb3JrIiwNCiAgICAic3RhdGUiOiAiTlkiLA0KICAgICJwb3N0YWxDb2RlIjogIjEwMDIxLTMxMDAiDQogIH0sDQogICJwaG9uZU51bWJlcnMiOiBbDQogICAgew0KICAgICAgInR5cGUiOiAiaG9tZSIsDQogICAgICAibnVtYmVyIjogIjIxMiA1NTUtMTIzNCINCiAgICB9LA0KICAgIHsNCiAgICAgICJ0eXBlIjogIm9mZmljZSIsDQogICAgICAibnVtYmVyIjogIjY0NiA1NTUtNDU2NyINCiAgICB9LA0KICAgIHsNCiAgICAgICJ0eXBlIjogIm1vYmlsZSIsDQogICAgICAibnVtYmVyIjogIjEyMyA0NTYtNzg5MCINCiAgICB9DQogIF0sDQogICJjaGlsZHJlbiI6IFtdLA0KICAic3BvdXNlIjogbnVsbA0KfQ0KYGBgDQoNCg0KIyMgRXhhbXBsZQ0KDQojIyMgT01EQiBhcGkgDQpsaWtlIGh0dHA6Ly9pbWRiLmNvbS8NCg0KIyMgUiBQYWNrYWdlcyAtLSBSZWxhdGVkDQoNCipQZW9wbGUgd2hvIHVzZSBKU09ObGl0ZSBhbHNvIHVzZS4uLioNCg0KKiBodHRyIC0tIGNhbGxzIEpTT05saXRlIGluIHNlcnZpY2UgdG8gbWFqb3IgZ29hbCBvZiBtYW5hZ2luZyBIVFRQIA0KKiBydmVzdCAtLSAgdXNlZCBmb3IgaHRtbCBwYXJzaW5nDQoNCg0KDQo=